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Abstract 

We consider a discrete population of users with homogeneous service de¬ 
mand who need to decide when to arrive to a system in which the service 
rate deteriorates linearly with the number of users in the system. The users 
have heterogeneous desired departure times from the system, and their goal 
is to minimize a weighted sum of the travel time and square deviation from 
the desired departure times. Users join the system sequentially, according 
to the order of their desired departure times. We model this scenario as a 
non-cooperative game in which each user selects his actual arrival time. We 
present explicit equilibria solutions for a two-user example, namely the Sub¬ 
game Perfect and Cournot Nash equilibria and show that multiple equilibria 
may exist. We further explain why a general solution for any number of users 
is computationally challenging. The difficulty lies in the fact that the objective 
functions are piecewise-convex, i.e., non-smooth and non-convex. As a result, 
the minimization of the costs relies on checking all arrival and departure order 
permutations, which is exponentially large with respect to the population size. 
Instead we propose an iterated best-response algorithm which can be efficiently 
studied numerically. Finally, we compare the equilibrium arrival profiles to a 
socially optimal solution and discuss the implications. 


1 Introduction 

The strategic timing of arrivals to congested systems is relevant for various appli¬ 
cations such as traffic, queueing and communication networks. We study a non- 
cooperative game in which atomic users need to time their arrival to a deterministic 
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processor sharing system with linear slowdown. This may be the case on a ring road 
around a business district in which the density at any point on the road affects all of 
the road, and therefore arriving users can cause a slowdown even for users who arrived 
before them. Throughout the paper we refer to the model as a traffic network where 
users travel along a route at varying speeds. Nevertheless, our aim here is to provide 
a general analysis of the strategic arrival times to such a processor sharing system, 
and the results are not limited to the specific traffic application. The linear slowdown 
dynamic can be seen as a discrete variation of the Greenshield’s fluid model (see for 
example [8]). This work complements [12] where the socially optimal arrival schedule 
of users to the same system was analysed. In this paper the choice of arrival times is 
made by the users themselves sequentially, according to the their desired departure 
times. Note that while all users are served simultaneously, the model presented here 
still maintains the First-In-First-Out property, and thus in the sequential game users 
leave the system in the same order they arrived. 

The study of departure time choice to a congested bottleneck goes back to Vickery 
[14], where a fluid queue dynamic was assumed. The research of fluid bottleneck 
models has evolved greatly since then, and we refer the reader to Arnott et ah [1] 
and de Dios Ortuzar and Willumsen [2] and references therein. Otsubo and Rapoport 
analysed a non-fluid (atomic user) game with discrete arrival instances in [10]. An 
arrival time and route choice (dynamic user equilibrium) game for a route with linear 
slowdown was analysed using a mean held approach by Mahmassani and Herman in 
[ 8 ]. 

A queueing theory approach to the strategic timinig of arrivals to a congested 
stochastic queue was developed by Glazer and Hassin in [3]. They introduced a 
game in which a discrete population of users, of a Poisson distributed size, choose 
arrival times to a single server exponential queue with the goal of minimizing waiting 
times. This led to another branch of research that relies on the stochastic properties 
of the queues, rather than huid dynamics. Examples with a discrete deterministic 
population, as we assume in this work, are the works of Juneja and Shimkin in 
[6] (tardniess costs), Ravner [11] (order penalties), and Haviv and Ravner [4] (loss 
system). All of the above assume random memoryless service times and a hrst come 
hrst served regime. An arrival time game to a processor sharing queue was studied 
by Raheja and Juneja in [5], using a fluid approximation. In the queueing context 
our work is the first to analyse an arrival time game with a discrete population queue 
with heterogeneous users and deterministic service times. 

2 Traffic model 

In this section we introduce the model and show that despite its seeming simplicity, 
it in fact yields a complex arrival-departure dynamic. Suppose a set of atomic users 
M '.= {1,...,V} need to travel on a single route of length 1. We define the travel 
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speed on a segment at time t as: 

v{t) := p - a{q{t)-1), (1) 

where q{t) is the number of users on the segment at time t, /? > 0 is the free flow speed 
of a single user travelling alone, and a > 0 is the slowdown parameter. We assume 
that /3 — a{N — 1) > 0, in other words, this means that the travel speed is positive 
even if all users travel at the same time. Note that for a > ^, the only possible case is 
N = 2. In Figure 1 the system dynamics are illustrated as a function of the number of 
concurrent users. Observe that while travel speed decreases by dehnition, the overall 
service rate is non-monotone and concave. In particular, it is initially increasing, has 
a maximal throughput at level q = and then decreases to almost zero when the 
system is very busy. 
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Figure 1: The individual travel speed and aggregate throughput as the number of users in 
the system increases. Parameter values: (3 = 1 and a = 0.01 


Every user i E JV has a desired departure time from the system, denoted by d*. 
Without loss of generality we assume that the desired departure times are ordered: 
d* < d*, \li < j. The action of user i is choosing an arrival time Oj G R. Denote the 
arrival and departure vectors of all users by a := (oi,..., a^v) and d := (di, ..., dAr), 
respectively. The cost incurred by user i is 

Ci{a) = {di-d*f +'-i{di-at). ( 2 ) 

This cost function is a combination of a quadratic penalty for any deviation from the 
desired departure time, be it early or late, and a linear penalty for the total travel 
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time. We focus on this cost function for the sake of a clear presentation, but all of 
our analysis can be applied to any convex function (of the deviation and travel time 
terms) in a straightforward manner. We further discuss on how this generalization 
can be made in the concluding remarks in Section 7. The minimal cost of any user 
is and can only be obtained by travelling alone at free flow speed and leaving at 
exactly the desired time, d* for user i. 

The effective departure times of users are determined by a and the travel dynamics 
dehned in (1): 

rdi 

1 = / V (t) dt, i G A/", (3) 

J ai 

where q{t) = l{te[ai,di]}- Using (1) we get a set of N equations for d. 


I = {di- ai){(3 + a) - a 


f*di 


di 


jGAT 


l{t E [ttj, dj]}dt, 


eAf. 


These N equations can be treated as equations for the unknowns d, given a or vice- 
versa. In [12] it was shown that the departure dynamics for an ordered vector a are 
given by 

D d — Aa = 1, 

where A G and D G are dehned as follows: 

' 

(3-a{i-hi), i = j 

i -|- 1 < j < fcj ) 

0, o.w. 


Aij : — 


—a, 


(3 - a{ki - i). 


Di, ■= 


-a, 


0 , 


with ki := max {/c G A/" : a*, < dj} and hi 
result of this is the recursive formula 


i = j 

hi < j < i — 1 ) 

o.w. 

:= min{/i G A/" : dh > Oj}. A direct 


di 


1 +{(3 - a{i - hi))ai +a 


spki 

Z-^j=i 


+1 


f3 — a{ki — i) 


i G X. 


(4) 


Using an iterative algorithm we can compute the unique d for any given a (or vice 
versa) with at most 2N computations. At this point it is important to observe that 
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the vector k ;= (fci,..., /cjv) defines the combined order permntation of all arrivals 
and departures. For example, if iV = 3, the prohle (oi < a 2 < di < < d 2 < ds) 

corresponds to k ;= (2, 3, 3). In this example there is an overlay between users 1 and 
2 during the interval [ 02 , di], and between users 2 and 3 during the interval [ 03 , d 2 ]. 
Therefore, given k we know the exact number of users and their travel speed at any 
point in time. Furthermore, as long as there is no change in the permutation, the 
departure times are continuous with the arrival times with a known linear coefficient, 
as shown in Figure 2. Note that (4) simply solves the traffic dynamics without any 
consideration of the cost function ( 2 ) that users wish to minimize. 

If we denote the set of all possible arrival-departure permutations by 

/C ;= {k G = N, ki < kj \/i < j} , 

/2N) 

then |/C| = This follows by observing that the elements of /C correspond uniquely 
to lattice paths in the N x N grid from bottom-left to top-right with up and right 
movements without crossing the diagonal. The number of such elements is the Catalan 
number (see [ 12 ] for more details). 

The relation between arrival and departure times, dehned in (3), is in fact a set 
of piecewise-linear equations. This is illustrated in Figure 2 by changing the arrival 
time of a single user while keeping all others hxed. 



Figure 2: Arrival-departure dynamies: An illustration of the effect of changing the 
arrival time of one user 1 (green user) while keeping the arrival times of the other three 
users (2,3,4) (blue) constant. The horizontal axis is the arrival time of the green user. 
The vertical axis shows the arrival and departure times of all other users: Arrival times of 
blue users are the blue dashed horizontal lines at (0.05,0.15,0.45) and the arrival time of 
the green user varies and is thus represented by the dashed diagonal line. Departure times 
are the solid lines. The numerous vertical dotted lines illustrate “break-points”. In between 
these lines, the effect of changing the green user’s arrival on all users is affine. The red 
vertical line marks a point where a departure overtakes an arrival, the yellow vertical line 
marks a point where an arrival overtakes an arrival and the black marks a point where an 
arrival overtakes a departure. The parameter values of the example are: a = 1.5 and /3 = 5. 
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The piecwise linear relation between arrival and departure times implies that the 
cost of any single user is not convex with respect to his own arrival time, even though 
the cost function, dehned in (2), has a convex form. This piecewise-convex behaviour 
is illustrated for the same numerical example in Figure 3. We can see that the cost of 
any user is not convex, monotone, or differentiable with respect to a change in arrival 
time of any user (including himself). Clearly, this complex cost structure leads to 
difficulties both in finding a socially optimal arrival schedule and a Nash equilibrium 
schedule. 



Figure 3: An illustration of the effect on the cost of changing the arrival time of a single 
user while keeping the arrival times of the other three users constant. The horizontal axis 
is the arrival time of user 1. The vertical axis shows the cost of all users. The parameter 
values of the example are: a = 1.5, /3 = 5, and d* = 0.5 for i = 1,..., 4. 

In [12] it was shown that in the socially optimal arrival schedule the order of 
the users is in accordance with their departure times. For small values of N (< 15) 
the exact socially optimal arrival scheduled can be computed using an exhaustive 
algorithm. For the general case, a heuristic search algorithm was suggested and shown 
to be efficient for larger values of N (< 500). The key underlying property which 
makes scheduling difficult using these travel dynamics is the fact that the departure 
and cost functions behave differently given the combined order permutation of the 
arrival and departure times. From a combinatorial perspective, the number of possible 
permutations is exponential with the number of users. Unsurprisingly, this property 
will make the game of arrivals, defined in the next section, computationally difficult 
to solve as well. 


3 Ordered arrival game 

The game of arrivals is defined as follows: every user i = 1,..., N chooses an arrival 
time tti G [aj_i,oo), where oq := —oo. Let a_i denote the arrival times vector of all 
users excluding i. The set of best response actions of user z G A/" to the arrival times 
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of all other users is 


BTZi{a._i) := argmin {ci{a,a_i)}. 

a£[ai—i,oo) 

Note that BTZ is a set that may contain multiple minimizers, as the cost function is 
not convex. 

We consider two possible game formulations. The hrst is a simultaneous game 
(Cournot) in which all users decide at the same time, without inspecting the arrival 
time of previous users. In this case we impose the order of arrivals on the game in 
the following manner: if user i selects time a* < aj for some i > j then his effective 
arrival time will be = aj. We limit our analysis to pure strategies throughout this 
work, although mixed strategy equilibria are possible and are likely to exist as well. 

Definition 3.1 A pure strategy Cournot Nash equilibrium (CNE) is a vector of or¬ 
dered arrivals, a G such that a* G B7l{a_i), \/i G M. 

An alternative, and perhaps more natural, formulation for this model is a se¬ 
quential game (Stackelberg), in which users make their decisions after observing the 
arrival times of the previous users. In this setting the action space of user i G A/” is 
a function, bi{ao, ■ ■ ■ ,ai-i) : 1R*“^ R, specifying his arrival time given the arrival 
time of all preceding users. The dehnition of the best response action is therefore 
not sufficient to dehne the equilibrium. We provide a dehnition using a constructive 
backward induction method. 

Definition 3.2 A pure strategy Subgame Perfect Nash equilibrium (SPNE) is a col¬ 
lection of functions, {bi : R*“^ ;—?■ R}, that satisfies the backward induction: 

• The initial condition is the function bjsi : R'^“^ —)■ R, defined by choosing an 
arbitrary element bjq G BlZ{ai, ..., otv-i) for any 

• Eor i < N, 


hi{ao, ..., Oi-i) G argmin {ci(ai,..., ai_i, a, Ri(a))}, 

aG[ai_i ,oo) 

where 

Bi{a) = (6i+i(ai,..., a^-i, a),...,bN (ai,..., a,..., bN-i{ai, ..., a,...))), 

is the collection of best response functions of users j > i. If the minimizer at 
any step is not unique then a minimizing element can be arbitrarily selected. 

Unfortunately, this method is only tractable for small values of N, as we will 
show in the next sections. The CNE is a more coarse equilibrium solution, as it is not 
subgame perfect. Nevertheless, it is a Nash equilibrium in both games that represents 
a stable schedule of arrivals in a sense that no single user has incentive to deviate. 
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Take care however, and note that a CNE is not necessarily an eqnilibrinm path of 
any SPNE. This is dne to the fact that a CNE may inclnde non-credible threats; 
the strategy instrncts a nser to play an action that is snb-optimal for some actions 
of the preceding nsers, bnt becanse all play a fixed action these sitnations are not 
realised. In the next section we shall show this can happen for a non-degenerate range 
of parameters in a two-nser example. 

Compnting a CNE is a more modest goal, bnt still compntationally challenging 
for games with a large nnmber of nsers. This will lead ns to snggest a henristic 
method to hnd snch eqnilibrinm paths in Section 5.1. Nnmerical analysis nsing the 
above method will be presented in Section 6, along with a comparison to the socially 
optimal solntion of [12]. Before proceeding to the general analysis for any nnmber of 
nsers we will commence by explicitly solving the two-nser game, which provides some 
important insight abont the model dynamics and difficnlties. 


4 Two-user game 

Snppose that there are only two nsers arriving to the system. We assnme that they 
both have the same desired departnre time and select it to be zero withont loss of 
generality, dl = d 2 = 0. We arbitrarily set nser 1 to be the leader. We first analyse 
the SPNE of the seqnential game, with the resnlts snmmarized in Proposition 4.1. 
The proof explicitly constrncts all possible eqnilibria nsing a backward indnction 
argnment. This will be followed by the compnting the CNE of the simnltaneons 
game, which is fnlly characterised in Proposition 4.2. 

From eqnation (4) we have that for oi < 02 the departnre times are given by: 


(ii(ai, 02 ) 


l-\-/3ai—aa2 

0—a. 


, Oi < 02 — 

, 02 — < Oi < 02, 


and 


(i2(fli) 0 - 2 ) 


02 + ;^ 1 0,1 < a2 — 

l+Pa2+a{di{ai,a2)-a2) „ \ ^ ^ „ 

- ^, 0.2 — p ^ 0.1 ^ 0 . 2 - 


Recall that the free flow speed is /3 and the travel time when a nser is alone in 
the system is Hence, the condition 02 > Oi ^ can be interpreted as: the second 
arrival takes place after the hrst departnre. 

A nniqne featnre of the two-nser model is that both nsers spend the same amonnt 
of time in the system. This is becanse dnring the time that both nsers are in the 
system they travel at the same speed, /3 — a, hence they travel the same distance 
dnring this time. For the remaining distance, which is identical for both, they travel 
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at the free flow speed and again reqnire the same time to complete the jonrney. 
Therefore, 


d2 — Oi2 — d\ — CLi 


1 

/3 


l—a{a 2 —ai) 

( 3 — 0 . 


1 ^1 ^ ^2 ^ •) 

, 0-2 ~ ^ ^ ^ ® 2 - 


The cost of nser 2 given that user 1 is arriving at oi is then given by (2): 


C 2 (ai, 02 ) 


J , a2>ai + 

f l+0a2+a{di{ai,a2)-a2)\‘^ , ,,l-a(a2-ai) „ ^ ^ „ i 1 

^- ) +T' —— ,ai<a2<ai + ^. 


If Oi < — I then user 2 can obtain a minimal cost of ^ by arriving at exactly — 
For this best response to be unique we need to assume 7 > 0. Otherwise, if 7 = 0 
and — 7 ^ < Oi < — I then user 2 can obtain a cost of zero by arriving at either — ^ 
or at — ^p32a both yield ^2 = 0. For instance, ai = 02 = — 7 ^ guarantees that 

both travel at the slowest speed and depart together at zero. 

If oi > — |, the minimal cost satisfying 02 > ai + ^ is obtained by 02 = ai + ^, 
i.e., arriving immediately upon user I’s departure, for arriving later will increase the 
tardiness cost and not change the travel time. For the second region, ai < 02 < ai + ^, 
the optimal arrival of user 2 is given by solving the corresponding one dimensional 
constrained quadratic program. After plugging in di( 01 , 02 ) and applying simple 
algebra we have that the quadratic program that needs to be solved is 

min [^x^U + xV+ W'\ , 

ai<x<ai + ^ 


where 


U 

V 


f / 3 - 2 a Y 

[/ 3 -aJ 


2(/3 — 2a)(l + aoi) — 'ya(/3 — a) 


and IF is a constant with no impact on the optimization. We denote the unconstrained 
minimum of the quadratic program by X = — The best response of user 2 to Oi 
can then be explicitly stated as 


62 ( 01 ) 


_i n <- _2 

/3 ) 01 

X*(Oi) , Oi > 


(5) 
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where 


a , a> X, 

x’^ia] := < X , X - ^ < a < X, 

0- + ^ , a < ^ 

Note that if a = | then U = 0 (the cost function is linear) and we dehne x*{a) := a+^- 
Furthermore, as explained above, if 7 = 0 then the best response is not unique for 


Oil G 


. Specihcally, arriving at yields a cost of zero, and so does 


0—a ’ 

arriving at — ^ as prescribed by (5). 

The optimal choice of user 1 if user 2 is playing 6 : IR —> IR is the collection of 
arrival times, 

61 (oo) = argmin{ci(a, 62(a))}, 

CL^CLq 


where ci is the cost function (2) and oq = —C )0 as in Dehnition 3.2. We are now ready 
to fully characterise the SPNE. 


Proposition 4.1 For N = 2 and d* = (0,0) the SPNE are characterised as follows. 


1. In any case, the strategy of user 2 is to arrive according to 62 (ai) for any arrival 
time of user 1, ai. 

2. 7/7 = 0 then 

(a) if a < ^ the unigue SPNE is ai = — 7 ^- 


(b) if a > ^ then there are two SPNE: ai = 
resulting equilibrium paths (realisations) are 


/S—Oi 

1 


and Oi = —The 


(3 — 0. ’ (3 —a 

with respective equilibrium costs (0, 0) and (^0, ^ j . 

3. 7/7 > 0 then there is a unique SPNE such that 

(a) if a < f thenai = argmin„>_^{ci(a, 62 (a))} G 

(b) if a > ^ then ai = — The resulting equilibrium path is 
costs 


and 




^,0j with 


1 J. 

/3’ 0 


, fl2 + 


Proof We have already established that 62 ( 01 ) is the best response strategy of user 2, 
hence this is the strategy played in any equilibrium. Computing the optimal response 
of user 1 to this strategy requires solving two quadratic programs: the hrst is a trivial 
one for oi < — |, and the second one gives x*. 
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If a > ^ then user 1 can make sure user 2 arrives at 0, resulting in the minimal cost 
of ^ for user 1. This is achieved by arriving at Oi = — which satishes Oi < X — 

If 7 = 0 then arriving at — also achieves the minimal cost for user 1, which is zero 
in this case. Therefore, if 7 = 0 both oi = — ^ and Oi = —^ 7 ^ are SPNE (together 
with 62 ( 0 ))- 

If a < I then there is a unique minimizer, —| < a < —to 01 ( 0 , 62 ( 0 ))- This is 
seen by considering the unconstrained quadratic program solution for user 2 , 

^ 2{f3 — 2a){l + aa) — 'ya{f3 — a) 

^ 2(/3 - 2 a )2 • 

If 7 = 0 then X < 0 for a = — ^, implying that 62 (-i) < 0. Thus, we have that 

oi = the unique SPNE in this case. If 7 > 0 then X is a monotone decreasing 

function of a, hence 62 ( 0 ) is monotone decreasing as well and ci(a, 62 ( 0 )) has a unique 
minimizer. | 

In Figure 4 we illustrate the realised arrival times of the two users according to 
an SPNE for varying values of the travel time penalty, 7, and the linear slowdown 
coefficient, a. 
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Figure 4- SPNE arrival times density for 7 € [0,8] with j3 = 1, d* = (0,0), and different 
values of a. 
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In order to find the Cournot Nash Equilibrium points, the best response of user 
1 for any arrival time of user 2 needs to be computed. Note that & 2 (ai) as defined 
above remains the best response for user 2 given any action oi of user 1 . As before, 
by considering the model dynamics we get that the best response is given by solving 
the respective quadratic program for each region of the cost function ( 2 ), 


Ci(ai, 02 ) 


f l+/?ai \ I 7 

1 , /3 ; 


/ l+/3ai—Qa2 

I /3-« 


)N 7 


l—a{a 2 —ai) 

(S—a 


, ai < a2 - 

, 0,2 — ^ ^ ttl ^ 02 - 


The unconstrained minimum of the quadratic cost in the second region is then 

_ 2/5(1 - 002 ) + 7 a (/9 - a) 

“ 2^2 • 

This yields the best response function for user 1, 


\ , 02 > 0 , 
[ y*{a2) , 02 < 0 , 


( 6 ) 


where 


y*{a) 


a , a < Y, 

< Y ,Y <a<Y + j, 

o — ^ ,o>y + ^. 


A CNE in this example is any pair of arrival times ai < 02 such that 02 G & 2 (oi) 
and ai G 61 ( 02 ). Note that the equilibrium path of an SPNE is not necessarily a CNE, 
as user 1 now minimizes his cost with respect to a single action of user 2 which does 
not change with his own action. Using similar arguments to those used in the proof 
of Proposition 4.1 we can obtain the following characterisation of the CNE points. 


Proposition 4.2 For N = 2 and d* = (0, 0) the CNE are characterised as follows. 
1. 7/7 = 0 then, 


(a) if a < ^ the unique CNE is 


(3—a ’ (3—ol ] ^ 

(b) if a> ^ then there are two CNE: ^ ^ ^ 

2. 7/7 > 0 then, 


j3—a. ’ 13—a 


and 


70). 


12 












(a) if a < ^ then, 

i- */ 7 < a(p^) then there is a unique CNE: 
1 70 ; (/9^ — Aafi) 


ai = 


1 7 a (/9 + 2a) 
0-2 = — 7 ;- 


(]-a 2p^{/3-2a) ’ " (3 - a 2/3(/3 - 2 a) ’ 

a. if'j > then any point ( 01 , 02 ) satisfying 


oi — 02 — —, 02 G 


mm 


7 a 1 


max < — 


7 a 


f3 2{(3 - 2a) 


,0 


is a CNE, 

(b) if a > ^ then any point ( 01 , 02 ) satisfying 


Oi — 02 — 02 G 




is o C'A^E. 


Proof Clearly, in equilibrium Oi < — ^ and < ^2 < 0, otherwise trivial re¬ 

ductions can be made to at least one of the cost functions. Plugging Y in the best 
response function ( 6 ) for the relevant range yields 


61 ( 02 ) 


a _ 1 _ 'raC-a) 
/3 “2 13 2/32 

« 2 - 7 


- ^ < «2 < -f, 

- i < «2 < 0 . 


The best response function (5) of user 2 depends on the values of 7 , a and /? in a 
slightly more elaborate manner. Specifically, if a > ^ then 


62 ( 01 ) = < 


1 

' h 


-ior 


Ol + ^ 


i 

' P 


Ol + ^ 


1 + QQl 

l3-2a 


, oi < -^,7 = 0, 


, - I < oi < -7,7 = 0, 

, Oi < -f,7 > 0, 


, oi > ^, 770 , 
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and a a < ^ then 


&2(ai) = < 




) ^ fj-i 


, - I < ai < -I + 


-ya 


(3-2a 


a „ 1 I 'ya(P-a) 

- 0-1 — a o„, T- 


p-2a ' 2(/3-2a)2 




j3 ' 2{0-2a) 


13 ' 2{(3-2a) 

^ < ai <-^ 


+ 


ya 


(3-a ' 2(/?-2a) ’ 


(l\ 


-^ + 


ya_ 


(3-a ' 2(/?-2a) 


< ai < 


1 

y- 


Note that some the regions in the latter fnnction may be empty, depending on the 
valne of 7 . 

1. If 7 = 0 then oi = 02 = is clearly an equilibrinm because both users incur 

the minimal cost of zero. Furthermore, in this case. 


61 (aj - y - i 


3 


a 


< 02 < 0 , 


which further implies that < &i(o 2 ) ^ ~y 

(a) If q; < I then /9 — 2q; > 0 and —| > —^7^, and thus & 2 (&i(o 2 )) = ^ 1 ( 02 ) 

for < ^ 1 ( 02 ) < — 7 - The only solution to 

Oi 1 

O2 = &2(&l(02)) = ^1(02) = -^02 — 
is then given by 02 = — 7 ^- 

(b) If a > ^ then there are two solutions to 02 = & 2 (&i(o 2 )): the hrst is in the 
second region of & 2 (oi), 


l + «(fa 2 -i) 

=- J^a -’ 

yielding oi = 02 = — 7 ^, and the second is 02 = oi + 7 = 0 in the third 
region of 62 ( 01 )- 


2. If 7 > 0 then oi = 02 cannot be an equilibrium. This is because 61(02) = 02 only 
in case 02 = — 7 ^, which is never a best response for user 2 if 7 > 0. Therefore, 
there are two possible types of equilibrium: 02 = oi + 7 (and Oi = 02 — 7), or 
the simultaneous solution to the equations ai = Y and 02 = X. 

(a) If a < I then. 
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7 — a{i 3 -'a) 02 = ai + ^ is a best response to Oi = 02 — ^ if 

2 2 ja 1 1 7 a 

-j<a,<-- + 2(^ _ 2„) ^ ^ + 2(/J-2a)’ 

but then 02 < —^7 (from 7 < ), which implies that oi = 02 — ^ 

is not a best response. Hence the unique CNE is given by the interior 
point (Y,X) as stated in the proposition, 
ii. If 7 > then there is no CNE in the interior. All pairs satisfying 

the condition in the proposition statement are such that best response 
of user 2 to oi = 02 — ^ is 02 = oi + ^, and vice versa. The interval for 
02 is obtained by considering the best response regions of 61(02) and 
62(01). 

(b) If a > I then 62(01) never has an interior solution, i.e. given by X. 
Therefore all equilibrium are of the form 02 = 01 + ^, and are given by the 
range specihed in the proposition statement. | 

Remark 4.3 The multiple equilibria, in fact a continuum of equilibria, arises in 
cases where interaction is too costly, except for the case 7 = 0 which yields a different 
type of non-unique solution. Specifically, the equilibrium is not unique when there is 
a steep slowdown in travel speed: a > ^, or when the travel time penalty 7 is very 
high. In all such equilibria the users travel alone at free flow speed. 

We computed the CNE arrival ranges for different parameter values, along with 
the realised arrival time corresponding to the SPNE. In Figure 5 a we set a < ^, and 
in this case the CNE are unique and close to the SPE actions for small 7, whereas for 
larger values of 7 there is an expanding range of possible CNE such that both users 
travel alone. In Figure 5 b we illustrate an example with a > |. In this case the CNE 
is never unique and can be quite far from the SPNE actions for larger values of 7. 
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di 



di 


- User 1 (SPNE) 

- User 2 (SPNE) 

. User 1 (CNE) 

. User 2 (CNE) 



(b) a = 0.6 


Figure 5: SPNE and CNE arrival times for 7 € [0, 8 ] with (3 = 1, and d* = (0,0). The solid 
lines are the SPNE actions, and the dotted lines and regions represent the range of possible 
CNE. 


In Figure 6 the user arrival times in equilibrium are compared with those specihed 
by the socially optimal prohle, 

{a°, 02 ) = argmin{ci(ai, 02 ) + 02 ( 01 , 02 )}, 

(01,02) 

which can be obtained as before by solving two two-dimensional quadratic programs 
and taking the minimum of both solutions. As expected, the SPNE arrival times are 
not optimal except for the extreme case of 7 = 0. Furthermore, if 7 is large enough 
then in all cases users travel on disjoint time intervals. Specihcally, according to the 
SPNE user 1 arrives at —1 and leaves at the desired time of 0, incurring just the travel 
time cost of 7 , while user 2 arrives at 0 and leaves at 1 , thus incurring a tardiness cost 
of 1, and a total cost of 7 -|-1. In the socially optimal prohle, still for large values of 7 , 
both users incur the same cost of 7 -|- 0.25 by arriving at —1.5 and —0.5, respectively. 
Note that it can easily be verihed that in the two-user game both users incur the 
same cost under the socially optimal prohle, for any value of 7 . The sequential form 
of the game gives user 1 the advantage of hrst action which allows him to travel alone 
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and arrive at his desired time, when 7 is large enongh. The socially optimal profile 
“forces” nser 1 to share the cost and arrive earlier than he wonld selfishly choose. 
The CNE displays very similar behavionr to the SPNE when 7 is small bnt as it gets 
bigger there are mnltiple eqnilibria that inclnde both the socially optimal and SPNE 
solntions. 


di 



- User 2 (SPNE) 

. User 2 (CNE) 

User 2 (OPT) 

- User 1 (SPNE) 

. User 1 (CNE) 

User 1 (OPT) 


Figure 6: SPNE, CNE and socially optimal arrival times for 7 G [0,8] with ft = I, a = 0.2, 
and d* = (0,0). The solid lines are the SPNE outcomes, the dashed lines are the optimal 
solutions, and the dotted lines and regions represent the range of possible CNE. 


5 The general arrival game 

When compnting the backward indnction or the CNE for the two-nser example we had 
to solve two qnadratic programs, one for every order of the pair ( 02 , di). We showed 
that already in this simple example the eqnilibrinm was not necessarily nniqne, in 
both game formnlations, and behaved differently for different parameter valnes. In 
order to compnte all CNE for any nnmber of nsers, iV > 2, we mnst solve a qnadratic 
program for every possible permntation of the departnres and arrivals. The backward 
indnction of the SPNE is even more demanding, as at every step i = 1,... ,N, we 
mnst take into acconnt the solntion of steps z +1,..., iV, which will yield general (not 
necessarily qnadratic) programs with respect to a*. In Section 2 we explained that 
the permntations can be represented by the vectors k G /C, and made reference to 
the fact that |/C| grows exponentially (specifically, as the Catalan nnmbers). Thns, 
even finding a single eqnilibrinm point, let alone characterising all eqnilibria, is not 
an easy task. This leads ns to snggest a henristic method to find CNE points. 

5.1 Ordered best response algorithm 

Best response dynamics are often nsefnl for several reasons. Most notably, becanse 
points of convergence are Nash eqnilibrinm points. Another advantage is that they 
are constrnctive and can provide a feasible evolntion of the behavionr of nsers leading 
to an eqnilibrinm solntion. In many types of games best response dynamics are known 
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to converge to a unique Nash equilibrium for any initial point. This is the case for 
example in potential games [9], and sub-modular games [13]. Unfortunately, our game 
is not an instance of any of these games. We already know that the equilibrium is 
not necessarily unique. Furthermore, the non-convex and non-smooth form of the 
cost function suggests there is not much hope for a constructive iterative procedure 
that is guaranteed to hnd an equilibrium point in polynomial time. Nevertheless, if 
we optimistically apply the best response procedure and it does indeed converge for 
a given initial point then the output is a CNE. 

We formally dehne the best-response algorithm starting at any ordered arrival 
prohle ao, denoted by BR(ao), as follows. 

Best response algorithm - BR(ao) 

(1) Initiate a := ao. 

(2) For i = 1,..., N, let Bi = BTZ{ai ,..., a^-i, hj+i,..., oat) and compute 




(3) If hj = Oj, \/i = 1,..., N then stop, otherwise set a* = a* and go back to step 


( 2 ). 


Remark 5.1 The rule of updating the arrival time of a single user when the mini- 
mizer is not unigue is defined in a way that guarantees changes will only occur when 
there is a strictly positive decrease in cost for the user. Specifically if di is in the best 
response group then we make no change, and otherwise we choose the smallest ele¬ 
ment, argminSj. This avoids the algorithm getting stuck in loops caused by multiple 
minimizers. Examples of such loops can easily be constructed. Note that this does not 
rule out other kinds of loops. We have not been able to construct other types of loops, 
nor have we been able to prove they don’t exist. 

Remark 5.2 The algorithm may step outside of the action space because the opti¬ 
mization for user i is on the interval [aj_i, cxd), regardless of the arrival times of user 
j = i 1,..., N. However, after a full iteration on all users the arrival profile will 
always be ordered. Furthermore, we argue that this is more natural than limiting the 
search to the interval [ai_i,aj+i], as the game is seguential and in the actual course 
of play the users have to react to the users preceding them. Note also that if we re¬ 
strict the order during the iteration then a point of convergence is not necessarily an 
eguilibrium. 
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In [12] a method for optimizing the sum of total costs with respect to a single 
coordinate, while keeping all others hxed, was presented. We will now detail how step 
(2) of the BR algorithm can be carried out using a modihcation of the aforementioned 
method. Let a be an ordered arrival time vector, and suppose we now want to 
minimize the cost function Cj(aj,a_i) by changing only a*, i.e., find the set 

argmin{(di(a, a_j) - d*+ 7 {di{a, a_j) - a*)}. 

Recall that the model dynamics yield a piecewise-quadratic behaviour of the cost 
function with respect to any single coordinate, as was illustrated by the solid line in 
Figure 3. Therefore, the minimizer can be found by computing the minimal value 
of a single variable quadratic function on every continuous segment. Every segment 
corresponds to an order of all arrival and departure times (indexed by k G /C) and 
the coefficients of the piecewise-linear term dj(a, a_j) are given by (4). The number 
of possible segments is given by the number of possible order changes. In Lemma 7 of 
[12] an upper bound of 0{N^) was given for the number of possible segments. This 
upper bound was obtained using worst-case analysis, and the number of computations 
was typically much less in practice. 

We have shown in Proposition 4.2 that there may exist multiple equilibria. More¬ 
over, the algorithm may not converge at all if there exist loops in the best-response 
dynamics. To address these problems we suggest running the algorithm on a set of 
initial prohles, denoted by Mo, which may be generated randomly or according to 
some specihed structure. In addition, we define a maximum iteration parameter that 
aborts the algorithm if it does not converge within the predefined number of itera¬ 
tions. We denote the repeated algorithm by RBR(Mo). This will also allow us to 
numerically analyse the price of anarchy which is defined as the ratio between the 
cost corresponding to the worst CNE and that of the socially optimal arrival profile. 
Formally, 

_ maXaeg^e^.^^Q(a^) 

where M® is the set of CNE and sl° is the socially optimal arrival profile. In the 
numerical analysis we substitute M® with the output of RBR(Mo). 


6 Numerical analysis 

Throughout this section we will compute equilibrium results using the BR and RBR 
algorithms and compare them with the socially optimal solution obtained using the 
algorithms presented in [12]. When possible {N < 15), the optimal solution is exact 
and otherwise it is a result of a polynomial-time search method that is guaranteed to 
converge to a local minimum. We use superscript notations of “o” and “e” to denote 
socially optimal and CNE, respectively. 
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In table 1 we summarize the convergence statistics of 100 instances of the RBR 
with randomized initial prohles, for different user population sizes. 


N 

Convergence % 

Mean # of Iterations 

Min # of Iterations 

Max # of Iterations 

^ of distinct CNE 

20 

100 % 

6.64 

5 

8 

2 

50 

100 % 

7.11 

6 

8 

2 

80 

100 % 

7.44 

7 

9 

5 


Table 1: Summary of 100 RBR instances for three examples with different population sizes, 
and parameters (3 = 1, a = 0.7-^, 7 = and d* equals the N quantiles of a normal 
distribution with mean 0 and variance 1. The random initial points where selected as follows: 
50 vectors were sampled from as the order statistic of an iid uniform distribution on (—1,1), 
and 50 vectors were sampled from a normal distribution with mean d* — ^ and variance 
between 0 and 2. 

All instances converged to equilibrium points within a small number of iterations. 
Multiple equilibria were found in all three examples, although the different CNE do 
not seem signihcantly different from each other. For example in the example of iV = 50 
the maximum absolute distance between any two equilibrium arrival schedules was 

N 

max } Itti — aA = 1.16, 

i=\ 

and the maximum distance between the arrival times of the same user in different 
equilibria was 

max max \ai — ad = 0.09. 

2 = 1 , 

To put these numbers in perspective, the free flow travel time in this example was 
1, the slowest travel time in a full system was ~ 4.3, and the time from the hrst 
arrival to the last departure in equilibrium was ~ 5. Therefore, a deviation of at 
most 0.1 between arrival times in all schedules suggests that all CNE schedules were 
effectively the same, and the small differences were perhaps due to the very non¬ 
smooth behaviour of the cost function. Recall that in Proposition 4.2 it was shown 
that for N = 2 inhnite equilibria may exist on a continuous interval, hence it is not 
surprising if this occurs in the general game as well for some small interval. 

Despite the positive convergence results demonstrated in the above examples, it 
should be noted that the scaling of the parameters is important. We ran the iV = 50 
example twice more, with a single parameter change each time, hrst we set a = 0.9^, 
and then we set d* as the N quantiles of a normal distribution with mean 0 and 
variance 0.1 {a was as in the original example). In both cases the algorithm did not 
converge on any of the 100 attempts after 100 iterations. In both of these examples 
the negative effects users incur on one another are very high. The lack of convergence 
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may indicate that the algorithm suffers from numerical precision difficulties when 
users want to arrive at virtually the same time, with respect to the congested travel 
speeds. If interaction costs are very high (not scaled) it is likely that there will be 
almost no overlap between users in equilibrium, and in this case other approximation 
methods, perhaps of a combinatorial nature, may be more suitable than the Best- 
Response algorithm presented here. 

Figure 7 shows the CNE and socially optimal departure times compared with the 
desired departure times for a specific set of parameters. In equilibrium users depart 
from the system closer to their desired departure time than under the socially optimal 
policy. This entails higher congestion costs and overall travel times. Specihcally, in 
this example the average cost for deviation from the desired time is 3.6 times higher 
under the socially optimal policy, but the average travel time costs are 3.8 times lower 
than in equilibrium. The individual user travel times can be seen in Figure 8. The 
price of anarchy in this example is 1.07. 


Departure time 
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-0.5 

-1 


• • 

• * 2 • 
i • * • 

« ! • • 



10 


15 


20 


User 


Figure 1: Desired, CNE, and socially optimal departure times for N = 20, ft = 1, a = 0.035, 
7 = 0.35, and d* equals the 20 quantiles of a normal distribution with mean 0 and variance 


0.16. 


Travel time 
9 



Figure 8: Individual user travel times of CNE and socially optimal profiles, for N = 20, 
ft = 1, a = 0.035, 7 = 0.35, and d* equals the 20 quantiles of a normal distribution with 
mean 0 and variance 0.16. 
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In Figure 9 we illustrate the individual user costs under both schemes. In equi¬ 
librium the cost is uni-modal with the users wishing to arrive in the middle of the 
interval incurring the most cost. The cost under the socially optimal policy appears 
bi-modal, with the cost more spread out. In Figure 10 we illustrate the smoothed 
queue size dynamics for both profiles. In equilibrium the system fills up faster, reaches 
a higher level, and is cleared earlier. This is because arrival times are less spread out 
in equilibrium. Hence, although users travel faster under the socially optimal policy, 
the last users will leave later simply because they arrived later than had they been 
free to choose their arrival time. 


Cost 


0.55 

0.5 



0.45 

1 5 




User 


15 


20 


Figure 9: Individual user costs of CNE and socially optimal profiles, for N = 20, ft = 1, 
a = 0.035, 7 = 0.35, and d* equals the 20 quantiles of a normal distribution with mean 0 
and variance 0.16. 


(lit) 



Figure 10: Smoothed queue size dynamics of the CNE and socially optimal profiles, for 
N = 20, 13 = 1, a = 0.035, 7 = 0.35, and d* equals the 20 quantiles of a normal distribution 
with mean 0 and variance 0.16. 

It should be highlighted that the patterns exhibited in Figures 7-10 are robust with 
respect to the game parameters. In particular, the difference between equilibrium and 
optimal arrival prohles were in the same direction for all instances tested, with varying 
effect sizes, and in this sense the examples presented here are a good representation 
of the general case. 
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7 Concluding remarks 

In this paper we have presented and analysed a novel ordered arrival time game to 
a congested processor sharing system, with a heterogeneous discrete user population. 
For a two-user example we provided explicit equilibrium analysis for two solution 
concepts, the Subgame Perferct Nash Equilibrium and the Cournot Nash Equilib¬ 
rium. This example highlighted the required steps for computing equilibrium points, 
and the resulting difficulties. It was shown that the ordered form of the game played 
an important role in both solutions. The two solutions display similar outcomes for 
certain game parameters, namely when interaction is not very costly. It was shown 
that there may exist multiple CNE, and that for some parameter values both the 
SPNE path and the socially optimal solution are CNE. For the general population 
model we presented a heuristic algorithm to compute CNE and analysed it numeri¬ 
cally. It was shown that this algorithm is very efficient if the parameters are scaled 
in a manner that does not allow the interaction costs to be too high. This allowed us 
to compare the equilibrium arrival process and the resulting congestion process with 
the corresponding socially optimal solution. The numerical analysis suggests that in 
equilibrium the travel times and queues are longer, but the users arrive closer to their 
desired times. This conclusion was supported by testing many population sizes and 
parameter values. 

There are several straightforward generalizations of the user cost functions that 
can be made. One can consider the general form: 

Ci(a) = fi [{di - d*Y) hi {{di - d* )+) -h gi{di - a^), 

where ft, gi, and hi are continuous convex functions for every i G TV. This allows for 
non-quadratic deviation penalties and a different penalty for being late or early. There 
is no substantial difference in the general model and all of our analysis still holds, with 
slight technical modifications. In particular, one needs to solve constrained convex 
programs instead of quadratic, and the differentiation in penalties for being late or 
early will lead to a larger number of piecewise-convex segments. The choice of the 
specific cost function in this work was made for the sake of brevity. A network arrival 
time game, in which users can choose arrival times and travel routes, may also be 
analysed using our model. The travel dynamics will be more complex when users 
travel on several subsequent segments with a linear slowdown, but can still be solved 
recursively using the framework we have provided. 

A non-straightforward generalization is allowing the users to have heterogeneous 
service demand, which translates to different travel distances in the traffic context. If 
this is the case then there is no longer a reason to order the arrivals according to their 
desired departure times, which adds a new layer of complexity to the analysis. It 
seems that there is not much hope for tractable solutions in this case and a different 
approach, perhaps combining a heuristic on order permutations, is called for. 

An additional open research question arising from this work is whether a fluid 
approximation, in the sense of [7], of the game can be solved. In the fluid version the 
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piecewise-linear dynamics will take the form of a set of retarded non-linear differential 
equations. It is unclear to the authors at this point if the corresponding fluid game can 
be solved directly or perhaps only by using a mean-held approach with less restrictive 
dynamics. 
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